A pp ea rs in A C L ’ 94 Similarity - Based Estimation of WordCooccurrence Probabilities

In many applications of natural language processing it is necessary to determine the likelihood of a given word combination. For example, a speech recognizer may need to determine which of the two word combinations \eat a peach" and \eat a beach" is more likely. Statistical NLP methods determine the likelihood of a word combination according to its frequency in a training corpus. However, the nature of language is such that many word combinations are infrequent and do not occur in a given corpus. In this work we propose a method for estimating the probability of such previously unseen word combinations using available information on \most similar" words. We describe a probabilistic word association model based on distribu-tional word similarity, and apply it to improving probability estimates for unseen word bigrams in a variant of Katz's back-oo model. The similarity-based method yields a 20% perplexity improvement in the prediction of unseen bigrams and statistically signiicant reductions in speech-recognition error.